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(54) Music apparatus with pitch shift of input voice dependentiy on timbre change 



(57) A music apparatus, a method of processing an 
input signal and a medium for use for it are provided 
r c iving an input signal composed of either of a voice 
signal and a tone signal and for processing the input 
signal based on a timbre change command signal to 
generate at least one channel of an output signal. In the 
music apparatus, a reference pitch designation section 
designates a reference pitch. An output signal genera- 
tion section receives the Input signal, the timbre change 
command signal and the reference pitch designated by 
the reference pitch designation section for changing a 
timbre of the input signal in accordance with the timbre 
change command signal, and for changing a pitch of the 



input signal above or below the reference pitch in 
accordance with the timbre change command signal, 
thereby generating the output signal having the 
changed timbre and the changed pitch. The output sig- 
nal generation section may change the pitch of the input 
signal above the reference pitch when the timbre of the 
input signal is changed by converting an original form- 
ant of the input signal to a female formant, and may 
change the pitch of the input signal below the reference 
pitch when the timbre of the input signal is changed by 
converting an original formant of the input signal to a 
male formant. 



FIG. 1 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to a process- 
ing apparatus of voice signal or tone signal for output- 
ting vocal hanmony. 

[0002] A processing apparatus for detecting, in real 
time, the pitch of a user's input voice signal (a lead voice 
signal), and for adding a harmonic voice signal to the 
voice signal to be output is well known and is described 
in Japanese Unexamined Patent Publication No. Hei 
11-133990. The pitch of the input voice signal is 
changed, and the resultant signal is output through a 
loudspeaker as a harmonic voice. At this time, various 
sound effects are added to the harmonic voice signal to 
provide a variety of harmonic voice variations. 
[0003] In order for this apparatus to be provided as 
a product, a further improvement is needed relative to 
the alteration of the sound quality of a lead voice signal, 
the alteration of sound quality and the conversion of the 
pitch for a harmonic voice signal, and the production of 
a user interface for easily performing the alterations and 
the pitch conversion and for applying sound effects. 

SUMMARY OF THE INVENTION 

[0004] To resolve the above problem, it is one 
obj ct of the present invention to provide a processing 
apparatus of voice signal or tone signal that can easily 
perform a clear timbre change for an input signal and 
perfomi various input signal pitch conversions, or can 
easily apply sound effects to an input signal. 
[0005] In a first aspect of the Invention, a music 
apparatus is constructed for receiving an input signal 
composed of either of a voice signal and a tone signal 
and for processing said input signal based on a timbre 
change command signal to generate at least one chan- 
n I of an output signal. The music apparatus comprises 
reference pitch designation means for designating a ref- 
erence pitch, and output signal generation means 
receptive of said input signal, said timbre change com- 
mand signal and said reference pitch designated by 
said reference pitch designation means for changing a 
timbre of said input signal in accordance with said tim- 
bre change command signal, and for changing a pitch of 
said input signal above or below said reference pitch in 
accordance with said timbre change command signal, 
thereby generating the output signal having the 
changed timbre and the changed pitch. Preferably, the 
output signal generation means changes the pitch of the 
input signal above the reference pitch when the timbre 
of the input signal is changed by converting an original 
formant of the input signal to a female fomiant, and the 
output signal generation means changes the pitch of the 
input signal below the reference pitch when the timbre 
of the input signal is changed by converting an original 
formant of the input signal to a mal formant. 



[0006] According to the first aspect of th present 
invention, since at the same time as the timbre of the 
input signal is changed, the pitch of the output signal is 
altered so that it Is higher or lower than the designated 

5 reference pitch, the change in the timbre is more easily 
discerned than it Is when the pitch of the Input signal Is 
adjusted to that of the ref r nee pitch. As an exampi of 
the clear diff r nee that can be provid d by the altera- 
tion of sound quality, when the quality of th input voice 

10 is changed to generate a lead or a harmonic voice sig- 
nal, by a formant conversion into female voice, the pitch 
of the input signal is raised until it is higher than the ref- 
erence pitch, while for a formant conversion into male 
voice, the pitch is reduced until it is lower than the refer- 

75 ence pitch. 

[0007] In a second aspect of the invention, a music 
apparatus is constructed for receiving an input signal 
composed of either of a voice signal and a tone signal 
and for processing said input signal in accordance with 

20 a timbre change command signal to generate at least 
one channel of an output signal. The music apparatus 
comprises pitch detection means for detecting a pitch of 
said input signal; and output signal generation means 
receptive of said input signal, said timbre change com- 

25 mand signal and said pitch of said input signal that is 
detected by said pitch detection means for changing a 
timbre of said input signal based on said timbre change 
command signal and for increasing or decreasing said 
pitch of said input signal based on said timbre change 

30 command signal, thereby generating said output signal 
having the changed timbre and the changed pitch. Pref- 
erably, the output signal generation means increases 
the pitch of the input signal when the timbre of the input 
signal is changed by converting an original fonnant of 

35 the input signal to a female fonnant, and the output sig- • 
nal generation means decreases the pitch of the input 
signal when the timbre of the input signal is changed by 
converting an original fonmant of the input signal to a 
male formant. 

40 [0008] According to the second aspect of the 
present invention, since the pitch of the input signal Is 
changed at the same time when the timbre is changed, 
the alteration of the timbre can be more clearly distin- 
guished. As an example of the clear difference that can 

4S be provided for the alteration of sound quality, when the 
quality of the input voice is altered to generate a lead 
voice signal or a harmonic voice signal, by formant con- 
version into female voice, the pitch is raised, while for 
formant conversion into male voice, the pitch is lowered. 

50 [0009] In a third aspect of the invention, a music 
apparatus is constructed for receiving an input signal 
composed of either of a voice signal and a tone signal 
and for processing said input signal in accordance with 
a chord designation signal to generate at least one 

55 channel of an output signal. The music apparatus com- 
prises a pitch conversion table stored for use in conver- 
sion of a pitch according to a chord, pitch detemriination 
means receptive of at least the chord designation signal 
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which designates a chord for referring to said pitch con- 
version table to determine a pitch of said output signal 
based on the designated chord, and output signal gen- 
eration means receptive of said input signal for chang- 
ing a pitch of said input signal to the pitch determined by 5 
said pitch determination means thereby generating said 
output signal having the determined pitch. Preferably, 
the music apparatus comprises a plurality of pitch con- 
version tabies corresponding to a plurality of harmony 
types which can be selected to determine a particular 10 
harmonic relation between said input signal and said 
output signal, wherein said pitch determination means 
refers to a pitch conversion table corresponding to the 
selected harmony type to determine a pitch of said out- 
put signal, and said output signal generation means 75 
generates said output signal having the determined 
pitch in parallel to said input signal to establish the par- 
ticular harmonic relation therebetween. 
[0010] According to the third aspect of the present 
invention, even when many chords are designated, by 20 
using the pitch conversion table, only a simple structure 
is required to determine the pitches of a variety of har- 
monic voices. 

[0011] In a fourth aspect of the invention, a music 
apparatus is constructed for receiving an Input signal 25 
composed of either of a voice signal and a tone signal 
and for processing said input signal in accordance with 
a kit designation signal to generate at least one channel 
of an output signal. The music apparatus comprises 
memory means for storing a plurality of parameter kits, 30 
each of which is constituted by a plurality of parameters 
used for characterizing said output signal and each of 
which includes^at least a parameter used for controlling 
a pitch of said output signal, parameter output means 
receptive of said kit designation signal that designates 35 
one of the parameter kits for referring to said designated 
parameter kit to output therefrom at least said parame- 
ter used for controlling the pitch of said output signal, 
and output signal generation means for receiving said 
input signal and for changing a pitch of said input signal 4o 
based on at least said parameter that is output by said 
parameter output means, thereby generating said out- 
put signal having the changed pitch. Preferably, said 
memory means stores a plurality of parameter kits in 
corr spondence to a plurality of harmony modes includ- 45 
ing a vocoder harmony mode, a chordal harmony mode, 
a detune hamriony mode and a chromatic hanmony 
mode, each of which is used for characterizing a har- 
monic relation of said output signal to said input signal, 
said parameter output means refers to said designated so 
parameter kit to output therefrom said parameters used 
for controlling said output signal, and said output signal 
generation means generates said output signal in paral- 
I I to said input signal to establish the harmonic relation 
therebetween according to the designated parameter 55 
kit. 

[0012] According to the fourth aspect of the present 
invention, since the parameters that characterize the 



output signal, such as the pitch of the output signal, can 
be collectively set by using the kit designation signal, a 
variety of parameter setups can be easily performed. 
[0013] In a fifth aspect of the invention, a music 
apparatus is constructed for receiving an input signal 
composed of either of a voice signal and a tone signal 
and for processing said input signal to generate at least 
one channel of an output signal. The music apparatus 
comprises effect setting means for setting parameters 
that are related to one or more sound effects to be 
applied to said output signal, effect instruction means 
for instructing application of at least one of said sound 
effects, and effect applying means operative based on 
said parameters that are set by said effect setting 
means and that are related to said sound effect for 
processing said input signal to generate said output sig- 
nal applied with said sound effect that is designated by 
said effect instruction means. Preferably, said effect 
instruction means is manually operable to instruct appli- 
cation of a sound effect to said output signal independ- 
ently from said input signal, and said effect applying 
means generates said output signal in parallel to said 
input signal while applying said sound effect designated 
by said effect instruction means to said output signal 
independently from said input signal. 
[0014] According to the fifth aspect of the present 
invention, without changing the setup of the effect set- 
ting means, whether a desired sound effect is to be 
applied or not can be determined simply by touching the 
effect instruction means. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0015] 

Rg. 1 is a functional block diagram for explaining a 
voice signal/tone signal processing apparatus 
according to one embodiment of the present inven- 
tion. 

Fig. 2 is a diagram for explaining the music play per- 
formed by the voice signal/tone signal processing 
apparatus of Fig. 1 , 

Rg. 3 is a diagram for explaining a lead voice that is 
generated by the voice signalAone signal process- 
ing apparatus of Rg. 1 . 

Rg. 4 is a diagram for explaining an example proc- 
ess performed by a formant modifier and a pitch 
shifter shown in Fig. 1 . 

Rg. 5 is another diagram for explaining the example 
process performed by the fornnant modifier and the 
pitch shifter shown in Fig. 1. 
Rg. 6 is a diagram for explaining a harmony mode. 
Rg. 7 is a diagram for explaining the types of voco- 
der hanmony modes. 

Rg. 8 is a diagram for explaining types of detune 
harmony modes. 

Rg. 9 is a diagram for explaining types of chromatic 
harmony modes. 
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Fig. 1 0 is a diagram for explaining types of chordal 
harmony modes. 

Hg. 1 1 is a diagram for explaining contents of a 
conversion table for tone names used in the chordal 
harmony modes. ^ 
Fig. 12 is a diagram for explaining parameters used 
by the voice signalAone signal processing appara- 
tus of Rg.i, 

Fig. 13 is a diagram showing harmony kits. 
Rg. 1 4 is another diagram showing harmony kits. w 
Fig. 15 is a diagram illustrating hardware anange- 
ment of the voice signal/tone signal processing 
apparatus of Fig. 1 according to the embodiment of 
the present invention. 

Rg. 1 6 is a diagram illustrating an external appear- is 
ance of the voice signalAone signal processing 
apparatus of Rg. 1 . 

Rg. 17 is a flowchart showing main processing and 
inten-upt processing. 

Rg. 1 8 is a flowchart showing panel setting process 20 
of Rg. 17. 

Rg- 19 is a flowchart showing process at step S66 
of Rg. 1 8. 

Rg. 20 is a flowchart showing process at step S53 
of Fig. 18. 



DETAILED DESCRIPTION OF EMBODIMENTS 

[0016] Rg. 1 Is a functional block diagram for 
explaining a voice signalAone signal processing appara- 30 
tus according to one embodiment of the present inven- 
tion. The overall arrangement will now be described. 
[0017] In Rg. 1. reference numeral 1 denotes a 
microphone, used as an input voice unit; 2. a keyboard 
by which play data are input by the depression of keys;' 35 
3. an automatic player, for reading stored play data; 4, 
an external input unit, for receiving MIDI (Muscal Instru- 
ment Digital Interface) signals; 5, an operation panel, for 
s tting functions or parameters; and 6. a pitch detector 
for detecting the pitch of input vofce (hereinafter refen-ed 40 
to as a vocal pitch), 

[0018] Reference numeral 7 is a formant modifier 
device for controlling the quality of Input voice. For 
example, reference numeral 7a denotes a switch for 
determining whether an input voice is to be passed 45 
while unchanged; 7b. a first formant modifier for chang- 
rng the formant of either a lead voice or a harmonic 
voice; and 7c and 7d. second and third formant modifi- 
ers for changing the formant of a harmonic voice The 
operations of the first to the third formant modifiers 7b to so 
7d include passive one wherein the operation is halted 
and no formant change is effected. 
[0019] Reference numeral 8 denotes a pitch shifter 
device for changing the pitch of an input signal, and ref- 
er nee numerals 8a to 8c denote first to third pitch shift- 55 
ers. For example, the first pitch shifter 8a changes the 
pitches of either lead voices or hannonic voices, and the 
second and third pitch shifters 8b and 8c change the 



pitches of harmonic voices. 

[0020] Reference numeral 9 denotes a pitch con- 
troller for using the pitch of the input voice received from 
the pitch detector 6. or the pitch of play data that is 
received from a channel allocator 10 to control the 
pitches of the signals that are received by the pitch 
shifter device 8 and a tone generator 12. Reference 
numeral 10 denotes a channel allocator for selectively 
allocating, as input controls forthe pitch controller 9 and 
the tone generator 12, the Input controls via the key- 
board 2. the automatic player 3 and the external input 
unit 4. Reference numeral 1 1 denotes a function con- 
troller for the overall control of the Individual functional 
blocks, and 12, a tone generator for generating a music 
tone signal. 

[0021] Reference numeral 13 denotes an effector 
device, and 13a to 13e. first to fifth effectore. The first 
effector 13a provides sound effects for lead voices, the 
second effector 13b provides sound effects for lead 
voices or for harmonic voices, the third and fourth effec- 
tors provide sound effects for hamionic voices, and the 
fifth effector 13e provides sound effects for musical 
tones. 

[0022] Reference numeral 1 4 denotes a signal out- 
put controller device, which is controlled by the function 
controller 1 1 . Reference numeral 14a to 14e denote first 
to fifth signal output controllers. The first signal output 
controller 14a controls volume ratios relative to the lead 
voice, the second signal output controller 14b controls 
volume ratios relative to either lead voice or harmonic 
voices, the third and fourth signal output controllers 14c 
and 14d control volume ratios relative to hannonic 
voices, and the fifth signal output controller I4e controls 
volume ratios relative to musical tones. Further, whether 
the individual signal channels are to be output is also 
determined. A harmonic voice signal is output with a 
lead voice signal output by either the signal output con- 
troller 14a or I4d. Further, a hannonic voice signal can 
be output independently, without a lead vofee signal 
being output. 

[0023] Reference numeral 15 denotes a pan con- 
troller; 16. an amplifier for mixing and amplifying the out- 
puts of the first to fifth signal output controllers 14a to 
14e and for outputting a stereo or 3D sound voice sig- 
nals or tone signals; 1 7, one or more loudspeakers; and 
1 8. a liquid crystal display device on an operating pknel 
[0024] The outline of the operation for this embodi- 
ment will now be described. The output of the micro- 
phone 1 is transmitted to the formant modifier device 7 
and the pitch detector 6. The exemplified formant modi- 
fier device 7 can output a maximum of four channels: 
one channel is for outputting an unchanged voice that is 
input and three channels are for changing the formants 
of input voices and outputting the results. When the 
input voice is not unchanged by turning off the switch 
7a. the first formant modifier 7b may change the formant 
of the lead voice. In this case, two channels of harmonic 
voices are output 
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[0025] The outputs of the first to third formant mod- 
ifiers 7b to 7d are transmitted to the first to third pitch 
shifters 8a to Sc. Sound effects are provided by the first 
to the fourth effectors 13a to 13c for the output of the 
switch 7a, the outputs of the first to third pitch shifters 8a 
to 8c, and the individual output channels of the tone 
generator 12. Further, the first to fifth signal processors 
1 4a to 14e output only a specific one or more channels, 
and the pan controller 15 performs weighting (control of 
a mixture ratio) to determine the localization of each of 
the signal channels. The output of the signal output con- 
troller 14a serves as a lead voice signal; the output of 
the signal output controller 14b serves as either a lead 
voice signal or a harmonic voice signal; the outputs of 
the signal output controllers 14c and 14d serve as har- 
monic voice signals; and the output of the signal output 
controller 14e serves as a music tone signal. These sig- 
nals are mixed by the amplifier 16. and the resultant sig- 
nal is released through the loudspeaker 1 7. 
[0026] The pitch detector 6 detects a vocal pitch by 
using a well known technique, such as zero-oross. in a 
voice analysis field, and outputs the vocal pitch to the 
pitch controller 9. Based on tire vocal pitch, etc., the 
pitch controller 9 cateulates the pitch after the formant 
conv rsion, and outputs it to the pitch shifter device 8, 
the formant modifier device 7. the tone generator 12 
and the effector device 13. Depending on the mode that 
is set, the pitch controller 9 calculates the pitch by using 
only the pitch of a harmony part that Is output by the 
channel allocator 1 0. 

[0027] - While a specific control mode for the pitch 
shifter device 8 will be described later, the pitch control- 
ler 9 has a function whereby control of the formant mod- 
ifier device 7 and the effector device 13 is exercised, 
and a function whereby the type of sound effect (includ- 
ing the sound quality) that is to be applied to a harmonic 
voice is changed, and/or the degree of a sound effect is 
changed in accordance with a pitch difference between 
the vocal pitch of the input voice and a harmonic voice 
whos pitch is changed. As a result, upon receiving the 
voice produced by a user, a variety of sound effects can 
be applied to a harmonic voice, or an appropriate sound 
effect in consonance with a pitch difference for the pitch 
of the user's voice can be automatically applied to a har- 
monic voice. 

[0028] The channel allocator 1 0 assigns, as a har- 
mony part, a signal received from the keyboard 2, the 
automatic player 3 or the external input unit 4, and out- 
puts it to the pitch controller 9, as is described above. 
Also, the channel allocator 10 allocates other play data 
to a musical tone channel, and controls the pitch of a 
musical tone that is generated by the tone generator 1 2. 
[0029] The output of the operation panel 5 controls, 
via the function controller 11. the functions of the form- 
ant modifier device 7, the pitch controller 9, the channel 
allocator 10, the tone g nerator 12. the effector device 
13. the signal output controller device 14, the pan con- 
troller 15, the amplifier 16, and the display device 18. . 



[0030] With the above describ d arrangement and 
operation, a desirable sound effect is applied to a lead 
voice that corresponds to a voice signal input at the 
microphone 1, a harmonic voice that is generated 

5 based on the input voice, and a musical tone, and at 
least one of these tones^ is selected and Is released after 
the mixing has been performed. As will be described 
later while referring to Fig. 12, etc., the sound effects to 
be provided include gender (the type and the depth of 

10 sound quality, such as a male voice, a female voice or a 
neutral voice), vibrato (the change ratio of the depth of a 
vibrato to a vibrato cyde, and the delay time before the 
vibrato starts), tremolo, volume, pan (localization), 
detune (detune of a harmonic voice in a mode other 

75 than a detuning harmonic mode, which will be 
described later), reverberation, or chorus. 
[0031] In order to easily understand the functions, 
in Fig. 1 , the effector is in charge of the application of a 
sound effect; however, a sound effect used for changing 

20 the pitch, such as vibrato or detune, can be provided at 
the same time as the pitch is changed by the pitch 
shifter device 8. The volume control and the pan can be 
performed by the signal output controller 14. The effect 
of gender is provided by the formant modifier device 7. 

25 [0032] The operation panel 5 and the function con- 
troller 1 1 are so designed that a sound effect to be 
applied to a lead voice signal that corresponds to a 
voice signal provided by a user, and a sound effect to be 
applied to a harmonk: voice signal can be independently 

30 set. Therefore, the user can employ the formant modi- 
fier device 7 and the sound effector device 13 to set 
mutually different sound effects, e.g., to set different 
types of sound effects or to set different intensities for 
the sound effect. For example, the depth of a sound 

35 effect to be applied to a harmonic vobe signal can be 
greater than the depth of a sound effect for a lead voice 
signal, or a random pan can be performed for a har- 
monic voice signal, while the localization of a sound 
image is not changed for a lead voice signal. 

40 [0033] Furthermore, in the default state, the func- 
tion controller 1 1 permits the formant modifier device 7 
and the effector device 1 3 to constantly provide different 
sound effects for a lead voice signal and a harmonic 
voice signal. As a result, a clear hanmonic voice can be 

45 generated for the original voice produced by a user. 
[0034] In the illustrated example, a total of four 
channels are provided as the channels of lead voice 
and harmonic voice signals. The number of signal chan- 
nels may be decreased or incraased. A lead voice may 

50 be transmitted to the firat signal output controller 14a 
without changing the formant or without applying any 
sound effect. The first formant modifier 7b, the first pitch 
shifter 8a. the second effector 13b and the second sig- 
nal output controller 14b may be defined as constituting 

55 a special block for processing a lead voice signal. In this 
case, the system constituted by the switch 7a, the first 
effector 13a and the first signal output controller 14a is 
• not required. The signal output controller device 14 can 
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select one or more arbitrary signal channels from 
among a lead voice signal, a plurality of harmonic voice 
signals and a musical tone signal, and can transmit 
them to the amplifier 16, and then they are released 
through the loudspeaker 7. 

[0035] Since the analog signal processing and the 
digital signal processing are not discriminated in the 
functional block diagram, an A/D converter and a D/A 
converter are not shown. As an example, an analog sig- 
nal entered at the microphone 1 is converted into a dig- 
ital signal by the A/D converter, and the digital signal is 
transmitted to the subsequent blocks. The signal output 
controller 14 weights a plurality of outputs, adds alto- 
gether their digital values, and outputs the result to the 
amplifier 1 6 via the D/A converter 
[0036] Fig. 2 is a diagram for explaining the music 
play operation performed by the voice signal/tone signal 
processing apparatus in Rg. 1. Fig. 2(a) is a diagram for 
explaining parts that are performed in an automatic 
accompaniment mode (style mode): and Fig. 2(b) is a 
diagram for explaining parts that are performed in an 
automatic play mode (song mode). In either mode, the 
vocal hannony is output. The vocal harmony is provided 
by the input voice part that is entered at the microphone 
1 and by the hannony part that serves as the playing 
input for a hannony part altogether or independent of 
the input voice part. 

[0037] In Rg. 1 described above, the allocation of 
the parts is established using the operation panel 5, and 
is performed by the channel allocator 10. which is con- 
troll d by the function controller 11. 
[0038] Rg. 3 is a diagram for explaining a lead voice 
that is generated by the voice signal/tone signal 
processing apparatus in Fig, 1 . Conventionally, in princi- 
ple, a sound effect Is created with a lead voice signal, 
while the pitch of an input voce entered at the micro- 
phone 1 is not changed. As a result, when the formant - • 
Is changed, the gender (sound quality) of the lead voice 
signal may be changed. However, merely by changing a 
formant, it is difficult to provide a clear aural change, 
such as from a male voice to a female voice. 
[0039] Therefore, when a gender change is desig- 
nated, the pitch of a lead voice signal is so altered that 
an appropriate gender result is provided, or the result 
falls within an appropriate range. As is illustrated, if a 
female voice is designated when the pitch of the input 
voice (vocal pitch) is substantially C3. the input voice is 
transposed +1 octave, and the lead voice signal is out- 
put while the obtained C4 is defined as the play data. 
When a male voice is designated, the input voice is 
transposed -1 octave, and the lead voice signal is output 
while the obtained C2 is defined as the play data. 
[0040] The transposition span Is not fixed to ±1 
octave, and may be ±3 or ±5 degrees. As well as the 
change in the sound quality, the transposition span 
(pitch shift distance) can be changed by the operation 
panels. 

[0041] For the vocal pitch of the lead voice signal, 



when pitch connection is designated, a vocal note is cal- 
culated whose pitch is nearest to the vocal pitch as a 
result of a comparison of wav lengths, and the pitch of 
the vocal note is obtained. Similarly when pitch con-ec- 
5 tion is designated at the time of a transposition, the 
transposed pitch Is rounded off for assignment of a spe- 
cific pitch name. 

[0042] The fomnant change and the pitch conver- 
sion of the lead voice signal described above are 
70 respectively performed by the formant modifier 7b and 
the pitch shifter device 8 In Fig. 1 . At this time, the switch 
7a is in the OFF state. 

[0043] In the above explanation, when a change of 
gender is designated, the pitch of a lead voice signal is 
15 changed by using, as a reference pitch, the pitch of the 
voice that is input (vocal pitch). However, if a melody 
channel is set, and if play data are input for a part (may 
not be merely a part entered at a keyboard, but may 
also be a part in a song track) that is allocated as the 
20 melody channel, the pitch of the lead voice signal is 
detemnined according to the pftch of the playing input for 
the melody channel. Therefore, when a change of gen- 
der is designated, the pitch is also transposed positively 
or negatively while the pitch of the playing input for the 
25 melody channel is used as a reference pitch. As a 
result, the change in the sound quality can be made 
clearer than when the pitch of the playing input for the 
melody channel is used as the pitch of the lead voice. 
[0044] A method for a formant change and a pitch 
30 conversion of a lead voice signal will be briefly 
explained. For a harmonic voice signal, the formant 
change and the pitch conversion are performed In the 
same manner. 

[0045] Fig. 4 is a first diagram for explaining an 
35 example processing perfonned by the formant modifier 
device 7 and the pitch shifter device 8 in Fig. 1 . m this 
diagram, the fundamental cycle of an output voice sig- 
nal is longer than the fundamental cycle of an input 
voice signal. In Rg. 4(a) Is shown an input voice signal 
40 wavefomn; in Fig. 4(b) is shown an input voice signal 
that has been extracted; in Rg. 4(c) is shown a window 
function; and in Rg. 4(d) is shown an output voice sig- 
nal. 

[0046] A phonemic segment is extracted from an 
45 input voice signal, and is extended or compressed to 
change the fonnant. In addition, a phonemic segment is 
inserted at the pitch interval of the lead voice signal to 
change the formant and the pitch, 
[0047] In accordance with the fundamental cycle of 
50 the input voice signal obtained by the pitch detector 6 in 
Fig, 1, the input voice signal is extracted and is multi- 
plied by the window function. While the waveform 
obtained by multiplication is employed as the element, 
the voice signal is arranged and is output in accordance 
55 with a desired fundamental cycle, so that an output 
voice signal having the altered pitch is obtained while 
the formant of the voice signal input is maintained. The 
extraction width is set. for example, to twice of the f un- 
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damental cycle of the voice signal input. 
[0048] To extract the input voice signal, the signal is 
temporarily stored in a memory and a predetermined 
extraction range is read therefrom. If the reading speed 
is higher than the writing speed, the waveform can be 
compressed. As a result, the formant is shifted to a high 
tone range, and the voice signal input, which has the 
sound quality of a male voice, can be changed so it has 
the sound quality of a female voice. The sound quality of 
the voice signal when originally input can be that of a 
female voice, and in this case, the fonnriant is shifted to a 
higher tone range, so that the sound quality is regarded 
as having been changed to a female voice. When the 
reading speed is lower than the writing speed, the wave- 
form can be extended when the voice signal is 
extracted. As a result, the formant will be shifted to a 
lower tone range, and the sound quality, which is repre- 
sentative of a female voice, can be changed to that of a 
male voice. 

[0049] Fig. 5 is a second diagram for explaining 
another example processing performed by the formant 
modifier device 7 and the pitch shifter device 8 in Fig. 1 . 
In this example, the cycle of an voice signal when it is 
output is shorter than the cyde that corresponds to the 
extraction width, Including a case wherein the cycle of 
the voice signal that is output is shorter than the funda- 
mental cycle of the input voice signal. 
[0050] In Fig. 5(a) is shown a voice signal output as 
the first channel (FaderO), which is the same as the 
voice signal output in Fig. 4(d); and in Rg. 5(b) is shown 
a voice signal that is extracted with a delay equivalent to 
the desired fundamental cycle of the voice signal that is 
output and is multiplied by the window function. This 
signal is defined as an output tone signal of the second 
channel (Faderl). When the first and the second chan- 
nels are combined, a voice signal output with the altered 
pitch can be obtained while maintaining the formant. 
[0051] In Fig. 5, as in Fig. 4, if the waveform is com- 
pressed when the voice signal input is extracted, the 
formant is shifted to a high tone range, so that the sound 
quality of the voice signal is changed to that of a female 
voice. If the waveform is extended, the formant is shifted 
to a low tone range, so that the sound quality of the 
voice signal is changed to that of a male voice. 
[0052] In Fig. 12 illustrating parameters, which will 
be described later, a parameter for a lead voice signal is 
listed. "Lead gender type" is a parameter for changing 
the sound quality, as described above. When the lead 
gender type is "ofT or "unisCon)," the formant is not 
changed. When the lead gender type is ''male,'' the 
formant is shifted to a low tone range, and when the 
lead gender type is ■fem(ale)," the fonmant is changed to 
a high tone range, it should be noted that the sound 
quality of ■unis(on)" can be changed by a parameter, 
"lead gender depth," that will be described later. 
[0053] Further, the pitch detector 6 can analyze the 
formant of the input voice to detect the sound quality of 
the voice. Whether the formant of the input voice should 



be changed to high or low or remain unchanged is also 
detemriined, so that the sound quality matches that set 
by using the operation panel 5. As a result, the sound 
quality can be set to the quality designated. 

5 [0054] The sound quality is not limited to the three 
levels of the male voice, female voice and neutral voice. 
More levels can be used for the formant change. In Rg. 
12, while three fonmant levels are employed, the inten- 
sity for the application of the gender effect is determined 

10 at multiple levels in accordance with the 'lead gender 
depth." For example, an extremely low voice or an 
extremely high voice can be set. Further, when the peak 
level of the fonmant differs, or the positions of a plurality 
of formant peaks are changed individually, such 

75 changes can provide a greater variety of sound quali- 
ties. 

[0055] Parameter "lead pitch correction" in Rg. 12 
is used to determine whether the pitch of a voice signal 
that has been input should be corrected to the nearest 

20 chromatic tone (a predetermined pitch tone determined 
by the pitch of a scale), or should be unchanged (free). 
By employing the pitch correction, the interval of an 
input voice signal that is deviated slightly can be 
changed to a con-ect interval. It should be noted that the 

25 parameter, "lead pitch connection," cannot be set in the 
"off" state of the "lead gender type" or in the detune har- 
mony mode. 

[0056] The parameter "Lead/harmonic balance" is 
for determining a volume balance between a lead voice 

30 signal (L). corresponding to the voice that is input, and a 
harmonic voice signal (H), "Lead vibrato," "lead vibrato 
depth" and "lead vibrato delay" are parameters for 
respectively determining a vibrato speed (Hz), a vibrato 
depth (cent), and a delay time (sec) required for a lead 

35 voice signal before a vibrato is begun. The vibrato for a 
lead voice signal is actually controlled in accordance 
with values obtained by multiplying the values of the' 
"lead vibrato rate," the 'lead vibrato depth' and the 'lead 
vibrato delay" t>y 1/127 of the "vibrato rate," the "vibrato 

40 depth" and "vibrato delay" in Rg. 12. 

[0057] A harmonic voice will now be described. In 
Rg. 1 , a maximum of three voices are released for a 
harmonic voice. In the following example, however, the 
maximum number of voices for harmonic voices are 

45 defined as two, and in case of providing a gender effect 
for a lead voice, the maximum is defined as one. 
[0058] Fig. 6 is a diagram for explaining the har- 
monic modes. A "vocoder harmonic mode," a "chordal 
harmonic mode," a "detune harmonic mode." and a 

so "chromatic harmonic mode" are prepared, and each 
harmonic mode is sorted to one or more harmonic 
types. 

[0059] Fig. 7 is a diagram explaining the types of 
the vocoder harmonic mode. The vocoder harmonk: 
55 mode is a mode in which, when the keyboard is played 
while voice is entered, a harmonic voice is generated 
using the sound quality of the input voice and having a 
pitch comparable to that specified by the keyboard. In 
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accordance with the harmonic type, the harnnonic voice 
to be generated is shifted an octave away from the pitch 
of the harmony part, or is shifted (is automatically trans- 
posed) within a one-octave range wherein the pitch of 
the voice is in the center range. 

[0060] Rg. 8 is a diagram for explaining the types of 
the detune harmonic mode. A detune hanmonic mode is 
a mode in which the pitch of input voice Is shifted 
slightly, and the obtained voice is released in order to 
provide a chorus effect Since the pitch of the harmonic 
voice is determined in accordance with the detuning 
value and the input voice, It does not affect the scale of 
a narmony part, such as the scale of the keyboard. 
Although only one type is shown, a plurality of types can 
be set by changing the detuning value. 
[0061] Rg. 9 is a diagram for explaining the types of 
the chromatic harmonic mode. A chromatic harmonic 
mode is a mode in which a harmonic voice is released 
that is shifted a fixed pitch away from that of the input 
voic . Since the scale of the harmonic voice Is deter- 
mined in accordance with the pitch shift distance and 
the input voice, it does not affect the pitch of the har- 
mony part. The pitch shift distance is varied by changing 
the type. 

[0062] Rg. 1 0 is a diagram for explaining the types 
of chordal harmonic modes. A chorda! harmonic mode 
is a mode in which, for example, a chord entered by a 
keyboard is Identified, and a harmonic voice consonant 
with the chord is generated. Merely by entering a voice, 
a harmonic voice consonant having a designated chord 
can be generated. The types for providing various har- 
monic voices that match jazz or blues can be selected 
by changing the harmonic types. Further, a voice 1 or 
voice 2 can be selected, or a harmonic voice having a 
high pitch (voice 1 is high) or a low pitch (voice 1 is low) 
can be designated relative to the pitch of the input voice.' 
[0063] It should be noted that " voice 1 Is bass ' 
means that the root tone of a designated chord is 
defined as the pitch of a harmonic voice. A "unison" is 
selected from among a harmonic voice having a pitch 
that corresponds to the pitch of the input voice, and har- 
monic voices having pitches that are higher or lower 
than that pitch by one to several octaves. When the gen- 
der type of the lead voice is not "off," the harmonic voice 
2 is not released. 

[0064] Instead of the keyboard play part, the auto- 
matic play part or the part assigned to an external 
d vice may be designated to the hannony part. For 
example, when a stored song is selected and a chord 
change is present in this song, the pertinent chord is 
ntered so that a harmonk: consonant with the progress 
of the musk: can be provided. 

[0065] Thirty-seven chord types that are specified 
in the MIDI standards can be identified, and the pitch of 
the harmonic voice can be determined in accordance 
with the chord type and the pitch of the input votee 
(vocal note). In addition, since it is desired that the pitch 
of the harmony may vary in accordance with the har- 



monc type, there is no conversion formula that can be 
applied for any hanmonic voice pitch. In this embodi- 
ment, therefore, the harmonic type, the chord type and 
the pitch of the input voice are detected and entered, 
5 and under these three conditions a conversion table is 
examined In order to determine the pitch of at least one 
type of harmonic voice. 

[0066] Alternatively the conversion table that is pre- 
pared for each harmonic type is selected In accordance 

10 with the harmonic type and is examined, while the pitch 
of the input voice and the chord type are employed as 
the condition entries, so as to detemriine the pitch of the 
harmonio voice. A set of such conversion tables is 
stored i i a ROM (Read Only Memory) or an external 

IS storage device, so that various harmonic types can be 
easily added later, or a part of the harmonk: types can 
be easily deleted In advance in accordance with a prod- 
uct model. 

[0067] In either case, since there are many combi- 
ne nations of the harmonic types, input signal pitches and 
designated chords, it is diffrcult to calculate the pitch of 
an output signal having an altered pitch according to the 
conversion rule. However, when the conversion table is 
employed, the pitches of a variety of harmonic voices 

25 can be determined by using only a simple structure. 
[0068] Fig. 11 is a diagram for explaining example 
contents of a pitch conversion table used in the chordal 
harmonic mode. In Fig. 11(a) Is shown a conversion 
table for a chord type "Major" for a harmonic type "duet' • 

30 below". In Fig. 1 1(b) are shown chord types "Major" and 
"minor" for a harmonic type "jazz above & below." 
[0069] In the conversion table, the pitch name of the 
harmonic voice signal, and data that represent an 
octave that is transposed from the octave for the pitch 

35 name (vocal note) of the input voice, are stored for each 
pitch name (C to B) ("lead voice name" in Fig. 11) for 
one octave of the vocal note of the input voice and is 
used as a reference. A to G entered in the columns for 
voice 1 and voice 2, which represent the harmonic 

40 voices, are pitch names for one octave; a 0 on the right 
Indicates that it falls within the octave of the input voice; 
value -1 indicates the pitch name of an octave that is 
lower by one octave than the input voice; and value 1 
indicates the pitch name of an octave that is higher by 

45 one octave than the input voice. 

[0070] In the above explanation, the conversion 
table is examined by using, as a reference, the vocal 
notes of the input voice. However, while the gender of 
the lead voice is controlled, the pitch of the input voice 

so may be changed and the lead voice signal may be gen- 
erated. In this case, the conversion table is examined by 
using the pitch of the lead voice signal as a reference, 
and the pitch of the harmonic voice in the chordal mode 
Is detemriined. Here, the vocal note of the input vorce 

55 may also be used as a reference to examine the conver- 
sation table. 

[0071] The same method is applied for the pitch 
that is used as a reference in the previously mentioned 
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vocoder mode for the automatic transposition, and in 
the detune mode and the chromatic mode. That is. for 
the above described harmonic types, while taking into 
account the fact that the pitch of the input voice is 
changed during the gender control and a lead voice sig- 5 
nal is generated, a harmonic voice can be produced by 
using the pitch of the lead voice signal as a reference. 
[0072] In the above description, the harmonic voice 
In the chordal harmonic mode is added relative to the 
pitch of the input voice; however, another type of har- 10 
monic voices can be generated. That is. when the play 
data are input to the part that is allocated as the melody 
channel, the conversation table in the chordal harmontc 
mode is examined as is done for the pitch of the play 
data of the melody channel, or the pitch that is changed 15 
under the gender control (the conversion table is exam- 
ined by replacing the pitch of the input voice with the 
pitch of the melody channel), so that the harmonic voice 
signal can be produced. 

[0073] Fig. 12 is a diagram for explaining parame- 20 
ters that are used by the voice signal/tone signal 
processing apparatus in Fig. 1 . Since the parameters for 
a lead voice have been explained, mainly the parame- 
ters for a harmonic voice will be explained. The "Har- 
monic gender type" is a parameter for determining the 25 
sound quality of a harmonic voice. When the parameter 
"harmonic gender type" is "off," the same sound quality 
is set as is set for the Input voice, and when the param- 
eter "harmonic gender type" is "auto," the sound quality 
of a harmonic voice is automatically changed in accord- 30 
ance with the following parameter. The "Auto upper gen- 
der threshold" Is used to detemnlne the number of semi- 
tones by which a harmonic voice must exceed the Input 
voice in order to start the harmonic gender control. The 
opposite parameter "auto lower gender threshold" is 35 
used to determine the number of semi-tones by which a 
harmonic voice falls below the input voice in order to 
start the hanmonic gender control. When the melody 
channel is designated and play data are entered for the 
part that is assigned as the melody channel, the sound 40 
quality is automatically changed, whiles rather than the 
pitch of the input voice, the pitch of the play data for the 
melody channel is used as the reference pitch. 
[0074] The "Upper gender depth" Is used to set the 
degree of conversion of a harmonic voice that exceeds 45 
the "auto upper gender threshold" to produce a female 
voic (although it sounds unnatural, this harmonic voice 
can be converted to produce a male voice in order to 
provide special sound effects). The "tower gender 
depth" is used to set the degree of conversion of a har- so 
monic voice that exceeds the "auto lower gender thresh- 
old" to produce a male voice (although it sounds 
unnatural, this harmonic voice can be converted to pro- 
duce a female sound). As the value rises, the resultant 
tone increasingly resembles a female voice, and as the 55 
value descends, the resultant tone increasingly resem- 
bles a male voice. 

[0075] The "Harmonic vibrato rate," "harmonic 



vibrato depth' and 'harmonic vibrato delay" are param- 
eters for d termining, for a harmonic voice, the speed 
(Hz) of vibrato, the depth (cent) of vibrato and the delay 
time (sec) required before the vibrato starts. The vibrato 
of the harmonic voice is actually controlled in accord- 
ance with a value obtained by multiplying these param- 
eter values by 1/127. 

[0076] The "Detune modulation" is a parameter for 
determining all the harmonic voices. "Harmonicl 
detune" and "hanmonic2 detune" are employed for voice 
1 and voice 2 for each harmonic voice, and the actual 
detuning value for each harmonic voice is determined 
by multiplying the two parameter values by 1/127. The 
"Hamrionicl volume" and "harmonlc2 volume" are 
parameters for determining the volume of each har- 
monic voice. The actual volume is determined by multi- 
plying the parameter values by "lead/harmonic 
valance." The "Harmonicl pan" and "harmonic2 pan" 
are used to detemnine the localization of each harmontc 
voice. R denotes the right localization, and L denotes 
the left localization. The "Harmony part" is effective 
when the "harmonic mode" Is the vocoder harmonic 
mode, and Is used to detemnine the part of a keyboard 
that controls the hamrionic voice. The "upper" is used to 
determine the addition of a harmonic to the keyboard 
play performed on the right region of a split point of the 
keyboard, and the "lower* Is used to determine the addi- 
tion of a harmonic to the keyboard play performed on 
the left region. 

[0077] The "Pitch -to- note switch" is used to desig- 
nate the generation, at the pitch of the input voice, of a 
musical tone that has the timbre of a part (R1, R2 or 
Left) of the keyboard that is designated by the parame- 
ter "pitoh-to-note part." The "Harmonic additional rever- 
beration depth" and "harmonic additional chorus depth" 
are used to determine the depth of the reverberation 
effect and chorus effect that are provided exclusively for 
a harmonic voice. The "Variation parameter" is provided 
for each kit that has an extended harmonic mode and 
will be described later. When the variation switch is 
turned on, the value of the "variation parameter" is tem- 
porarily changed. This temporary parameter value is 
determined by the parameter "variation value." 
[0078] As described above, not only are many 
parameters provided, but also, these parameters are 
mutually related. Thus, it is almost impossible for a user 
to use the operation panel 5 to set the individual param- 
eter values. Thus, a vocal harmony is sorted to a plural- 
ity of characterizing types (consisting of lead voices and 
harmonic voices). When the operation panel 5 is used 
to designate a harmonic type, the vocal harmony type 
that is preset in the ROM. and almost all the parameters 
that are related to the lead voice signal and the har- 
monk: vok:e signal are collectively set to specific values 
that are appropriate for the designated type. The group 
of parameters that are collectively designated is defined 
as a harmony kit (hereinafter referred to simply as a kit). 
[0079] When the stored kit is selected, that kit is 
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read and parameters are selectively set, so that a voice 
signal that has been input can be processed, and a har- 
nnony having various complicated pitches and sound 
effects can be easily output. A f nnale voice duet, a 
mixed chorus, country music, jazz, a cappella, etc.. are 
prepared as kits, and unique vocal tones can be collec- 
tively set for them. Since a conventional chordal har- 
monic merely follows a designated chord, only a 
common harmonic is added, and characteristic music 
performance, such as country music or jazz, cannot be 
coped with. However, by using the above described kits, 
a variety of setups can be easily obtained. 
[0080] Rg. 13 is a first diagram for explaining har- 
mony kits, and Rg. 14 is a second diagram for explain- 
ing harmony kits. In the lists of example kits. 49 types 
are shown, and for each kit, a name that describes the 
characteristic of the type is provided. Example parame- 
ters that are set by selecting a kit are those shown in 
Rg. 12. It should be noted, however, that only one part 
of the parameter values are shown in Figs. 13 and 14. 
Further, a case wherein a harmony kit is not selected is 
provided as one of the types. 

[0081] For a kit, the number of hamrionles and the 
localization can also be selected by designating the har- 
monic type. Multiple parameters conceming gender 
control are also included, as well as parameters con- 
cerning the production of sound effects for a lead voice 
signal and a harmonic voice signal, and parameters 
concerning volume and volume balance. The parame- 
ters registered as a harmony kit are not always fixed val- 
ues, and the operation panel 5 can be used to change 
or slightly adjust the values of part of the parameters. 
[0082] In Figs, 13 and 14, there are multiple kits 
having the kit names that are associated with gender. In 
most of these kits, the harmonic gender type is set to 
"Auto." In the kits for which "Auto* is set. although not 
shown, the "upper gender depth" is set to a value for a 
female-like voice, and the "lower gender depth" is set to 
a valu for a male-like voice. Therefore, when the pitch 
of the harmonic voice is higher than the reference pitch 
(the pitch of the input voice or the pitch of the melody 
channel) and the pitch exceeds the predetennlned 
"auto upper gender threshold" (frequently 0). the sound 
quality is near that of a female voice. When the pitch of 
the harmonic voice is lower than the pitch of the input 
voice and falls below the predetermined "auto lower 
gender threshold" (frequently 0), the sound quality Is 
near that of a male voice. 

[0083] When a kit that represents a female voice is 
selected on the operation panel 5 in Fig. 1 . the pitch of 
the harmonic voice for which the formant has been con- 
verted to produce a female voice is actually higher than 
the pitch of the input voice. When a kit that represents a 
male voice is selected, the pitch of the hamrionic voice 
for which the formant has been converted to produce a 
male voice is actually lower than the pitch of the input 
votee. When a kit that represents a mixed chorus is 
selected, the pitch of the harmonic voice for which the 



formant has been converted to produce female voices is 
actually high r than the pitch of the input votee, and the 
pitch of the harmonic voice for which the fbmnant has 
been converted to produce male voices is actually lower 
5 than the pitch of the input voice. 

[0084] Especially in the chordal hamnonic mode, 
"Auto" is set for ail the kits having a kit name that is 
associated with a female voice or a mixed chorus, and 
"Above" is designated for voice 1 by the "harmonic type" 
10 parameter. Therefore, a harmonic voice having a pitch 
higher than the input voice Is always set. Further, "Auto" 
is set for most of the kits having a kit name that is asso- 
ciated with a male voice or a mixed chorus, and "Below" 
or "Bass" is designated for voice 1 by the "harmonic 
15 type" parameter. Therefore, a harmonic voice having a 
pitch lower than the input voice Is always set 
[0085] For a kit that has a kit name associated with 
a male voice or a mixed chorus and for which " Auto" is 
not set. a harmonic voice is generated without changing 
20 the formant of voice that is input 

[0086] As previously described, an arbitrary "varia- 
tion parameter" can be set for each kit, and when the 
variation switch is turned on, the parameter value can 
be changed to a designated value. If . as a variation 
25 parameter, a parameter conceming sound quality is 
changed, a remarkable variation can be added as a 
sound effect for a vocal tone. 

[0087] For example, "Auto" is set for a kit for which 
the "hannonic gender type" setting is "Off." Othenwise, 

30 for a kit in which the setting for the "harmonic gender 
type" is "Auto," the "upper gender depth" or the "lower 
gender depth" is set to reflect an extreme value in the 
same direction (toward the same sound quality), or rn 
the reverse direction (away from the sound quality). 

35 Similarly for a lead voice, the designation (off. a male 
voice, a female voice or a neutral voice) for the "lead 
gender type" is mutually changed, or the value of the 
"lead gender depth" is changed to reflect an extreme 
value. 

40 [0088] When one of the above described kits Is 
selected, parameters concerning the vocal harmony 
(the lead voice signal and a plurality of harmonic voice 
signal types) can be collectively designated. Not only 
the number of types (voices) of harmonic voice signals, 

45 but also pitches and sound qualities above or below 
those of the input voice can be set In addition, sound 
effects, such as reverberation or vibrato, can be applied 
to vocal harmonies (the lead voice signal and the har- 
monfc voice signal), separately from a music tone sig- 

50 nal. 

[0089] The addition of revertDeration and other 
sound effects can be selected simply by using buttons 
on the operation panel, which will be described later. As 
is described above, since the input vofce and a musical 
55 tone can be easily hancBed separately, switching in 
match with music performance can be easily obtained. 
Parameters for collectively setting sound effects for the 
musical tone signal, the lead voice signal and the har- 
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monic voice signal may be included in a kit, or may be 
designated by using the buttons on the operation panel. 
[0090] Rg. 15 is a diagram, according to the 
embodiment of the present invention, illustrating the 
hardware arrangement of the voice signal/tone signal 5 
processing apparatus in Fig. 1. The same reference 
numerals as in Fig. 1 are used to denote conresponding 
components, and no further explanation for them will be 
given. Reference numeral 21 denotes a line input unit; 
22, an interface; 23, a CPU bus; 24, RAM; 25. ROM; 26, lo 
a CPU; 27, a tone generator; 28, a DSP; 29, an external 
storage device; 30, an interface; and 31, an external 
input/output unit. 

[0091] A/D conversion is performed, via the analog 
input interface 22, for an input voice received through a 75 
microphone 1 and a line Input unit such as a CO player 
and a tape cassette player, and the results are transmit- 
ted to the CPU bus. A plural'ity of hardware units, such 
as the RAM 24, the ROM 25 and the CPU 26, are con- 
nected to the CPU bus 23, and a display device 18 dis- 20 
plays a setup menu for harmony kits and individual 
parameters. In the ROM 25 is stored the voice sig- 
nal/tone signal processing program of the present 
invention that is executed by the CPU 26. as well as 
waveform data, preset data such as kits, a parameter 25 
conversion table, and demonstration song data for auto- 
matic playing. In the RAM 24 are prepared a working 
area required for the execution of processes by the CPU 
26. and a buffer area for parameter editing. 
[0092] A. ROM cartridge or a flexible magnetic disk 30 
(FD) is employed as a recording medium for the exter- 
nal storage device 29, which can also serve as a stor- 
age unit of the automatic player 3 in Rg. 1 . Timbre data 
and song data are stored on the recording medium, and 
data that are not stored in the ROM 25 can be added. 35 
Further, song data can be recorded or reproduced by a 
r cording/reproduction apparatus. The interface 30 
includes a MIDI Input/output terminal or an RS232C ter- 
minal, and exchanges MIDI data with the external 
input/output device 31. which may be a MIDI device 40 
such as a MIDI keyboard sequencer, a special tone 
generator, or a personal computer. 
[0093] The tone generator 27, which does not 
always correspond to the functional block of the tone 
generator 12 in Rg. 1, receives a tone parameter from 4S 
the CPU 23 and generates a musical tone signal. The 
DSP 28, which is controlled by the CPU 26, performs 
formant alteration, pitch detection and pitch conversion 
for a voice signal entered at the microphone 1 or a tone 
signal input :aiong the line input 21. and provides a so 
sound effect such as reverberation or chorus, for the 
voice signal or the tone signal. At least a part of the 
functions of the tone generator 27 and the DSP 28 may 
be implemented by software that is executed by the 
CPU 26. The functions of the above described DSP 28 55 
may be distributed so that different DSPs are employed 
for pitch detection and pitch conversion for a signal of 
the input voice, and for the application of a sound effect 



of an output signal. The signal output by the DSP 28 is 
converted into an analog signal by a D/A converter (not 
shown), and the analog signal passes through the 
amplifier 16 and is released as a sound signal through 
the loudspeaker 1 7. 

[0094] The CPU 26 employs the RAM 24 or the 
ROM 25 to process a voice signal entered at the micro- 
phone 1 , operation data entered at a keyboard 2 or at 
an operation panel 5. and play data received from the 
external storage device 29 or the external input/output 
device 31; displays various setup menu screens on the 
display device 18; controls the tone generator 27, the 
DSP 28 and the amplifier 1 6 based on the processed 
play data; and outputs MIDI data externally via the inter- 
face 30. The play data can be stored as sequence data, 
which includes time interval data, in the external storage 
device 29, or in the external input/output device 31 . 
[0095] The voice signal/tone signal processing 
apparatus of this invention can be implemented by the 
special hardware configuration in Fig. 1 5. This appara- 
tus can be implemented by a general-purpose personal 
computer wherein a digital/analog converter (DAC) is 
mounted and a codec driver is installed, and whereiri 
the voice signalAone signal processing program Is exe- 
cuted by a CPU and an operating system (OS). The 
voice SignalAone signal processing program is supplied 
along a communication line, or on a recording medium 
M, such as a CD-ROM, and is installed on a magnetic 
hard disk. 

[0096] This recording medium M is stored with a 
voice signal/tone signal processing program for treating 
as an input signal a voice signal or a musical tone sig- 
nal, and for processing the input signal to generate at 
least one type of output signal. The following recording 
medium M is employed. 

[0097] First, the recording medium M is stored with 
a voice signal/tone signal processing program that per- 
mits a computer to function as: reference pitch designa- 
tion means; and output signal generation means, which 
receives an Input signal, a timber change designation 
signal and a reference pitch designated by the refer- 
ence pitch designation means, and while changing the 
timbre of the input signal in accordance with the timbre 
change designation signal, changes the pitch of the 
input signal, so the pitch is made higher or lower than 
the reference pitch in accordance with the timbre 
change designation signal, and generates an output 
signal. 

[0098] Second, the recording medium M Is one on 
which is stored a votee signalAone signal processing 
program that permits a computer to function as: pitch 
detection means, which detects the pitch of the input 
signal; and output signal generation means, which 
receives an Input signal, a timber change designation 
signal and the pitch of the input signal detected by the 
pitch detection means, and while changing the timbre of 
the input signal in accordance with the timbre change 
designation signal, raises or lowers the pitch of the input 
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Signal In accordance with the timbre change designa- 
tion signal, and generates an output signal. 
[0099] Third, the recording medium M is one on 
which is stored a voice signalAone signal processing 
program that pennits a computer to function as: pitch 5 
determination means, which determines the pitch of the 
output signal by referring to the pitch conversion table; 
and the output signal generation means, which, to gen- 
erate an output signal, receives an input signal and 
changes the pitch of the Input signal so the pitch equals io 
the pitch of the output signal determined by the pitch 
d t rmination means. 

[0100] Fourth, the recording medium M is one on 
which is stored a voice signalAone signal processing 
program that permits a computer to function as: param- 75 
eter output means, which stores a plurality of parameter 
kits, each of which is comprised of a plurality of param- 
eters that include, at least, a parameter for controlling 
the pitch of an output signal and that characterize the 
output signal, and which receives a kit designation sig- 20 
nal and refers to the parameter kit to output, at least, a 
parameter for controlling the pitch of the output signal; 
and output signal generation means, which receives^the 
input signal and changes the pitch of the input signal in 
accordance with, at least, the parameter output by the 25 
parameter output means and which generates an out- 
put signal. 

[0101] Fifth, the recording medium M is one on 

which is stored a voice signal/tone signal processing 
program that permits a computer to function as: effect 30 
s tting means, which sets a parameter concerning one 
or more sound effects to be applied to an output signal 
for a vofce signalAone signal processing apparatus that 
employs a voice signal or a tone signal as an input sig- 
nal and that processes the input signal to generate at 35 
least one type of output signal; effect instruction means 
for instructing the application of at least one of the 
-sound effects to be provided; and effect applying means 
for s tting the sound effect based on the parameter that 
is set by the effect setting means and that is related to 4o 
the sound effect. 

[0102] Fig. 16 is a diagram showing the external 
appearance of the voice signalAone signal processing 
apparatus in Fig. 1 according to the embodiment of the 
present invention. The same reference numerals as in 45 
Figs. 1 and 15 are used to denote corresponding com- 
ponents, and no further explanation will be given for 
them. Reference numeral 41 denotes the main body of 
an electronic musical instrument; 42, an operator group; 
1 7A, a left loudspeaker; and 1 7B. a right loudspeaker. 50 
[0103] The main body 41 of the electronic musical 
instrum nt includes the keyboard 3 and the loudspeak- 
ers 1 7A and 1 7B. The operator group 42, which is com- 
prised of a plurality of operators, and the display device 
are provided on the operation panel 5. The keyboard 55 
and the operators are conceptually shown, and sp cific 
shapes and numtjers are not illustrated. Switches that 
are closely related to the present invention are an 



ON/OFF switch used to designate the output of vocal 
harmony (a lead voice signal and a hamrionic voice sig- 
nal); an ON/OFF switch used to designate the applica- 
tion of reverberation for the vocal harmony; and an 
ON/OFF switch used to designate the application of a 
sound effect other than the reverberation for the vocal 
harmony. In addition, there are an ON/OFF switch for 
designating the application of a sound effect for a musi- 
cal tone signal; a vocal harmony switch for designating 
a vocal harmony; a 'BACK" switch for changing a setup 
menu; a "NEXT" switch; and a switch and a "-" 
switch for selecting parameters. 

[01 04] Although not shown, the main body 41 of the 
electronic musical instrument includes a ROM car- 
tridge, an FD insertion slot, a MIDI terminal and an 
RS232C terminal. A pitch bend wheel and a modulation 
wheel may also be provided. 

[0105] The pan controller 15 in Fig, 1 , which deter- 
mines the localization of a sound image, controls the 
volume ratio of voices and musical tones that are output 
through the left loudspeaker 17A and the right loud- 
speaker 1 7B, so as to adjust the individual localized 
positions of input vocal tones, harmonic voices and 
musical tones. The pan control is also provided as one 
of the sound effects. Conventionally, random pan for 
randomly localizing musical tone signals is perfomied 
as one type of acoustic effect For example, while a user 
depresses a key, a musical tone signals are released in 
every direction, from the right and then from the left. A 
parameter may be included for applying this random 
pan individually to voice signals or to musical tone sig- 
nals. 

[0106] Figs. 17 to 20 are flowcharts showing the 
processing steps according to the embodiment of the 
present invention for explaining the operation perfomried 
by the voice signalAone signal processing apparatus. 
[0107] Fig. 1 7 is a flowchart of a main process and 
an intenrupt process. At step S51 , the apparatus is Ini- 
tialized, and at step S52, the operator group 42 is 
employed to input various control entries and to set var- 
ious parameters, while switching the screen of the dis- 
play device 1 8. This step will be described later, while 
referring to Figs. 1 8 and 1 9. At step S53, play data are 
detected, and a voice signal or a tone signal is proc- 
essed. This step will be described later, while referring 
to Fig. 20. 

[0108] At step S54, based on the various control 
entries and the parameters that are set, a lead voice, a 
harmonic voice and a musical tone are released. That 
is, based on play data corresponding to the depression 
of a key at the keyboard 2, the automatic play data 
received from the external storage device 29. MIDI data 
entered by the external input unit 4, or the voice signal 
or the tone signal entered by the line input unit 21 , those 
of a lead voice signal, a harmonic voice signal and a 
musical tone signal are generated in accordance with a 
control mode and parameters that are selected at the 
operation panel 5, and these signals are transmitted to 



12 

:OCID: <EP 1065651 A1J_> 



23 



EP 1 065 651 A1 



24 



the amplifier 16. 

[0109] For a vcx:al tone signal that is formed of a 
lead voice signal and a hanmonic voice signal, the play 
data entered at the keyboard can be employed to 
change not only an original voice signal that is input, but s 
also the timbre of the voice. Specifically, the gender of 
the sound quality can be changed (from a female voice 
to a male voice, from a male voice to a female voice, 
etc.), or the pitch can be altered. When the process at 
S54 is temriinated, program control returns to S52. and w 
the processes at steps S52 through S54 are repeated. 
[0110] Fig. 18 is a flowchart showing the panel set- 
ting process in Fig. 1 7, At S61 , a check is performed to 
determine whether an automatic accompaniment mode 
has been selected (setup is changed or execution is is 
instructed) using the operation panel 5. If the automatic 
accompaniment mode has been selected, program con- 
trol advances to S62. If the automatic accompaniment 
mode has not been selected, program control is shifted 
to S63. At S62, in accordance with the selection, the 20 
automatic accompaniment style, the ON/OFF state of 
the automatic accompaniment and the start/stop of the 
automatic accompaniment are designated in addition to 
other setups. Thereafter, program control retums to the 
main flowchart in Fig. 1 7. 25 
[0111] When the automatic accompaniment is 
played in the chordal harmonic mode, at the time when 
the automatic accompaniment begins for a musical 
tone, the pitch of a harmonic voice can be determined in 
accordance with a chord that is generated based on a 30 
chord entered at the keyboard and that is detected for 
the automatic accompaniment, and in accordance with 
the pitch of the input voice. The chord part for the auto- 
matic accompaniment need only be designated as a 
harmony part. 35 
[0112] At step S63, a check is performed to deter- 
mine whether the automatic play mode has been 
selected (the setup has been changed or the execution 
has been instructed) at the operation panel 5. If the 
automatic play mode has been selected, program con- 40 
trol advances to S64. If the automatic play mode has not 
been selected, program control is shifted to S65. At step 
S64. in accordance with the selection, the name of a 
song recorded in the ROM 25 or the external storage 
device 29 in Rg. 15 is set, and the start/stop is desig- 45 
nated, as well as other setups. Program control thereaf- 
ter returns to the main flowchart in Fig. 1 7. 
[0113] The harmonic mode selection data and the 
data indicating a specific track recording pitch data for 
controlling a harmonic voice can be written in a song, so 
When these data are detected, the specific track can be 
designated as a harmony part. At the time of the auto- 
matic play of musical tones generated by the tone gen- 
erator 27, the pitch of a harmonic voice can be 
automatically set. 55 
[0114] Othenwise, when it is known that pitch data 
for controlling harmonic voices are recorded on a spe- 
cific track for songs that are produced by a certain com- 



pany, and it is determined that the copyright contained 
in the selected song identifies this company, the specific 
track can be designated as a harmony part. At the time 
when the automatic play of musical tones is begun, the 
pitch of a harmonic voce can be automatically set. For 
a song, a user can also perform a track re-designation 
in order to control the harmonic voice. 
[0115] At step S65, a check is perfomned to deter- 
mine whether the vocal harmony has been selected. 
When the vocal harmony has been selected, program 
control advances to S66. If the vocal hannony has not 
been selected, program control advances to S67. To 
change the various setups for the vocal harmony, the 
"vocal harmony" button is depressed. 
[0116] Fig. 1 9 is a flowchart showing the process at 
S66 in Rg. 18. Steps S66a to S66f are selectively 
changed by using the NEXT button and the BACK but- 
ton, and in accordance with the steps designated, as is 
indicated by 18a to 18f, the display screen of the display 
device 1 8 is sequentially changed. 
[0117] The steps in Rg. 19 are performed for set- 
ting a vocal harmorty using the menu display screen. A 
vocal harmony is selected while the -characteristic 
thereof is provided by various parameters. A menu 
setup screen using a tab-dialogue is shown as the dis- 
play screen, and in the example, seven tabs are pre- 
pared. Since a mouse pointer is not employed, 
switches, such as the "NEXT" button and the "BACK" 
button, on the operation panel are employed to select 
tabs and setup entries. As needed, characters or pic- 
tures (not shown) to provide input guidance are dis- 
played in a blank portion in the tab-dialogue box. 
[0118] As previously described, there are multiple 
types of parameters, and it is difficult to set the parame- 
ters one by one. Thus, a plurality of parameters for a 
vocal harmony are preset and provided in the form of a 
kit 

[0119] At step S66a, a vocal harmony kit Is 
selected. As is shown on the display screen 1 8a, the kit 
tab-dialogue box is displayed in the foreground. 49 
types of harmony kits are prepared as shown in Figs. 13 
and 14. Since the display screen is small, a part of 
these types, i.e., four types, are displayed, and hamnony 
kits on the display can be scrolled by using the "+•" but- 
ton and the "-' button, and a highlighted harmony kit can 
be sequentially changed. When the "NEXT" button or 
the "BACK" button is depressed, the highlighted "stand- 
ard duet" is selected and entered, and the step is 
switched to the preceding or succeeding selection step. 
[0120] At step S66b to S66f, a part of the parame- 
ters that are collectively set as a kit. or other parameters 
that cannot be set as a kit. is designated. At S66a. the 
display screen of the following setup menu is changed 
in accordanc with a selected kit, and only selectable 
parameters are displayed, or the display highlighting is 
inhibited for parameters that cannot be selected. 
[0121] At Step S66b, the lead gender type is 
selected to change the sound quality of a lead voice 
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(microphone entry). For example, the tones released 
are for a female voice, even though a man Is singing. As 
is shown on the display screen 18b, the tab dialogue 
box for the gender type is displayed in the foreground. 
•MALE" Indicates a male voice, "FEMALE" Indicates a 
female voice, "UNISON" Indicates the intenmedlate 
sound quality of th male voice "MALE" and th female 
voice "FEMALE," and "OFF" indicates there is no 
change of the sound quality. The sound quality can be 
changed by using the "+" button or the "-" button. When 
the "NEXT" button or the "BACK" button is depressed, 
the highlighted "MALE" (male voice) is selected and 
input as a parameter, and the step Is switched to the 
preceding or succeeding selection step. 
[0122] At step S66c, a check is perfomied to deter- 
mine whether pitch correction, which is a function per- 
formed to correct an original interval (a lead voice) that 
has been deviated even slightly. On the display screen 
1 8c, "ON" or "OFF" is selected by using the "+" button or 
the "-" button. It should be noted that "ON" is not dis- 
played when the harmony l<it of the detune harmonic 
mode (a mode that additionally provides a harmonic 
having an interval that is slightly shifted away from the 
pitch of the voice that is input) is selected at S66a. or 
when "OFF" is selected at S66b. 
[0123] With the pressing of the "NEXT button or 
the "BACK" button, the highlighted "OFF" is selected 
and entered as a parameter, and the step Is switched to 
th preceding or succeeding selection step. 
[0124] At step S66d. a check is performed to deter- 
mine whether pitch-to-note Is to be performed whereby 
the timbre of a musical Instrument can be released at 
the pitch of the voice that is input. On the display screen 
1 8d, "ON" or "OFF" is selected by using the "+" button or 
the "-" button. Otherwise. In order to designate a pitch 
shift distance as a parameter, the pitch shift distance Is 
displayed for selection on the display screen 1 8d. When 
the pitch shift distance has been determined, a musical 
ton having a high pitch (e.g., the pitch is shifted one 
octave) can be released when a low voice is received. 
When the "NEXT" button or the "BACK" button is 
depressed, the highlighted "OFF" is selected and is 
entered as a parameter, and the step is switched to the 
preceding or succeeding step. 

[0125] At step S66e, the hannony part is selected. 
Only when, at step S66a, a hamnony kit that belongs to 
the "vocoder harmonic type" is selected as a vocal har- 
mony kit, a setup other than "OFF" can be designated. 
As the "vocoder harmonic type." a hamrionic voice Is 
added to the pitch employed for the playing at the key- 
board, using the sound quality of the voice, or the sound 
qualrty obtained by changing the gender of the voice 
that is input. The "hannony part" is a parameter for des- 
ignating the part of the keyboard that detemriines the 
pitch of the harmony when the keyboard is played. 
[0126] The value "OFP on the display screen 18e 
Is used to indicate that no harmonic Is added to the key- 
board play; "UPPER" Is used to indicat the provision of 



a harmonic for the keyboard play on the right region of a 
split point of the keyboard; "LOWER" is used to indicate 
the provision of a harmonic for the keyboard play on the 
left region of th split point. These parameters are high- 
5 lighted by using the "+" button or the "-" button. When 
the "NEXT button or the "BACK" button Is depressed, 
the highlight d "OFF" is select d and ent red. and the 
st p Is switched to the pr ceding or succ eding sel c- 
tion step. 

10 [0127] At step S66f. when a song is reproduced in 
the automatic play mode (song mode) and when a har- 
monic is added with the sound quality of the Input voice 
or the sound quality obtained by changing the gender of 
the input voice, a particular track of the song is selected 

75 so that the play data recorded on the pertinent track are 
used to determine the pitch of the voice. On the display 
screen 18f, tracks "1" to "16" are highlighted by using 
the "+" button or the "-" button. When the "BACK" button 
is depressed, the highlighted track "1" is selected and 

20 entered as a parameter, and the step is switched to the 
preceding selection step S66e. In addition, when the 
"NEXT" button is depressed, the highlighted track "1" 

^ may be selected and entered as a parameter, and the 
step may be switched to the first selection step S66a. 

25 [0128] At steps S66b through S66f, one of a plural- 
ity of values is selected on the setup menu. However, 
parameters may be edited using a method whereby 
numbered key buttons are used to enter the numerical 
values of the parameters and even fine adjustments in 

30 the values can be made, as desired by a user. 

[01 29] Further, not only may a system be employed 
for controlling the pitch of one or more harmonic voice 
signals based on a play signal received from one har- 
mony part, but also a system may be employed for pro- 

35 viding a plurality of hannony parts and for individually 
controlling the pitches of a plurality of harmonic voces, 
or a system may be employed for controlling the pitch of 
one or more harmonic voice signals based on a play 
signal that is obtained by mixing play signals received 

40 from a plurality of harmony parts. 

[0130] An explanation will be given while again 
referring to Fig. 1 8. When the process at step S66 is ter- 
minated, program control returns to the main flowchart 
in Fig. 17. When, at step S6S. the vocal hannony has 

45 not been selected, program control advances to step 
S67. At step S67, a check is perfomied to determine 
whether the vocal harmony is set to the ON state or the 
OFF state. When the vocal harmony is designated, pro- 
gram control advances to step S68. When the vocal har- 

50 mony is not designated, program control advances to 
step S69. 

[0131] At step S67. whether the "vocal harmony" 
button is depressed is examined to determine whether 
the ON/OFF state of the vocal harmony has been 
55 selected. When the "vocal harmony" button has been 
depressed, program control advances to step S68. If 
the "vocal harmony" button has not been depressed, 
program control advances to step S69. At step S68. 
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each time the depression is detected, whether the vocal 
harmony (a lead voice signal and a harmonic voice sig- 
nal) should be output is determined, and program con- 
trol thereafter returns to the main flowchart in Fig. 17. 
[0132] At step S69, whether the reverberation but- s 
ton for the vocal hanmony (lead voice signal and har- 
monic voice signal) has been depressed is examined to 
detenmine whether the reverberation effect has been 
selected for the vocal hamnony. When the revertjeration 
button has been depressed, program control advances io 
the process to step S70. When the reverberation button 
has not been depressed, program control advances the 
process to step S71 . At step S70, each time the depres- 
sion of the button is detected, whether the reverberation 
effect should be added to the vocal harmony is deter- is 
mined, and program control returns to the main flow- 
chart in Fig. 17. The parameter related to the 
reverberation of the vocal harmony is either set at step 
S74, which will be described later, or is preset, and 
reverberation is added to a generated vocal harmony. 20 
[0133] The reverberation effect is set independently 
of the reverberation that is to be added to a musical tone 
signal, so that the harmonic voice can be clearly distin- 
guished from a musical tone. Further, since the 
ON/OFF state of the reverberation can be controlled by 25 
the depression of one button, the ON/OFF state of the 
effect can be easily set for a harmonic voice, independ- 
ently of a musical tone. Therefore, it is not necessary for 
the setup screen to be opened each time so as to 
change the reverberation parameter to a desired value 3o 
or 2 ro. Further, for effects other than the reverberation, 
their ON/OFF states can also be controlled independ- 
ently of the setup operation for the parameters. 
[0134] At step S71, check is made to ascertain 
whether another effect button for vocal harmony (lead 35 
voice signal and harmonic voice signal) has been 
depressed in order to determine whether the ON/OFF 
state of an effect other than reverberation has been set 
for the vocal harrnony. When another effect button for 
vocal harmony has been depressed, program control 40 
advances to step S72. When any other effect button has 
been depressed, program control advances to step 

573. At step S72, each time the depression of the but- 
ton is detected, the effect to be applied to the vocal har- 
mony is determined, and program control returns to the 4S 
main flowchart in Rg. 17. Sound effects other than the 
reverberation effect that is to be added to the vocal har- 
mony is either set at step S74. which will be described 
later, or is preset. 

[0135] At step S73, a check Is performed to deter- 50 
mine whether a sound effect has been set. If there is an 
entry for a sound effect, program control advances to 
step S74. if there no entry has been made for a sound 
effect, program control advances to step S75. At step 

574, a sound effect to be added to the vocal harmony (a 55 
lead voice signal and a harmonk: voice signal) and to 
other common musical tones is selected on the menu 
display screen (not shown). First, a part for setting the 



application of a. sound effect is selected from among a 
plurality of parts shown in Fig. 2. As the harmony part, a 
harmonic voice higher than the input voice, a lower har- 
monic voice, and a lead voice corresponding to the input 
voice may be individually designated. The sound effects 
include reverberation, chorus, vibrato and random pan, 
and a gender effect (the sound quality type for a lead 
voice is set at step S66b in Fig. 19, as previously 
described) is provided for a vocal harmony. The param- 
eters representing the magnitudes of the effects are 
also prepared as, for example, a harmony kit. In addi- 
tion, for at least one part of the parameters, the setting 
can be changed greatly, or the parameter values can be 
slightly adjusted. When the process is terminated, pro- 
gram control returns to the main flowchart in Fig. 1 7. 
[0136] At step S75, a check is performed to deter- 
mine whether other setup has been entered. If other 
setup has been entered, program control advances to 
step S76. If no other setup has been entered, program 
control returns to the main flowchart in Fig. 1 7. At step 
S76, for each part, other setup such as the timbre of a 
musical instrument (a voice change), the volume, a pan 
or an octave shift, is designated, and setup concerning 
the execution of the automatic accompaniment or the 
automatic play is perfonned. Program control thereafter 
returns to the main flowchart in Fig. 1 7. 
[01 37^ Fig. 20 is a flowchart showing the process at 
step S53 in Rg. 17. At step S81, a key depression sig- 
nal generated while a user is playing the keyboard is 
detected, and program control advances to step S82. 
Normally, the key depression signal is used as play data 
to designate the pitch, and is released as a musical tone 
signal. At step S82, for example, the play data that are 
stored in the SMF (Standard MIDI File) form in the stor- 
age device are read and detected, and program control 
advances to step 883. That is, the play data are 
detected after the automatic play has begun. The play 
data detected here are processed in the same manner 
as the play data detected at step 881 . 
[0138] At step 383. the MIDI play data from the 
sequencer, the personal computer or the electronic 
musical Instrument are received at the external input 
terminal, and are detected. Program control then 
advances to step 884. The play data detected here are 
processed in the same manner as the play data 
detected at step 881 . 

[01 39] At step S84, the pitch of a voice signal input 
by the microphone or along the line is detected, and 
program control then advances to step 885. At step 
885, a check is performed to determine whether auto- 
matic accompaniment for a musical tone, or a chordal 
harmonic mode for a harmonic voice has been desig- 
nated. When either one has been selected, program 
control advances to step 886. If neither one has been 
designated, program control advances to step 388. 
[0140] At step 886, th designated chord is 
detected from the play data for the part that is selected 
as the automate accompaniment At step 887, the 
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chord play data that correspond to the designated chord 
are automatically generated, and program control 
advances to step S88. 

[0141] At step S88, a musical tone signal is gener- 
ated in accordance with the play data that have been 
entered, and a lead voice signal and a hamnonic voice 
signal are produced in accordance with the voice that is 
input. Program control then advances to step S89. 
[0142] In :the automatic accompaniment mode, 
basically, the chordal harmonic mode is appropriate as 
a harmonic mode. At the time when both a musical tone 
signal based on the play data for the melody part and a 
lead voice signal based on the play data for the input 
voice are output, the tone signal and the harmonic voice 
signal are automatically played at a pitch consonant 
with a chord designated by a part selected as both the 
automatic accompaniment part and the harmony part 
At this time, if a gender change is designated for the 
lead voice signal, the sound quality of the lead voice sig- 
nal is changed (from a male to a female voice), and the 
pitch is also changed In accordance with the sound 
quality. If "auto" is set for the gender control of the har- 
monic voice signal, the sound quality of the harmonic 
voice signal is changed in accordance with the pitch dif- 
ference with the input voice. 

[0143] When the vocoder hanmonic mode is 
selected, and when one of the parts such as automatic 
play part, the external input part or the key part of the 
keyboard operator is selected as a harmony part, the 
pitch of the input voice at the microphone is changed to 
the pitch of the hannony part and is then released. 
Wh n a change of gender is designated, the sound 
qualrty of the harmonic voice is also changed. 
[0144] At step S89, when pitch-to-note is desig- 
nated, the pitch of a musical tone is detemnined based 
on the pitch of the input voice (at the same pitch or a 
pitch having a predetermined relationship), and a musi- 
cal tone signal is generated using the timbre designated 
for the musical tone. Even for a user who has a bass 
voice, so long as an octave shift is designated for the 
pitch of the input voice, a melody can be generated at a 
high pitch having the timbre of a piano. 
[0145] At step S90, the designated sound effect is 
provided and the waveform process is performed in 
accordance with other parameters. Program control 
thereafter returns to the main flowchart in Fig. 1 7. 
[0146] In the above explanation, a male voice, a 
female voice or a neutral voice is employed as an exam- 
ple sound quality: however, the sound quality is not lim- 
it d to a feature that sounds like a male voice, a female 
voice or a neutral voice. Further, in the explanation, the 
voice of a user has been employed as the input signal. 
However, the voice used may be the voice of an animal, 
or may be a musical tone signal. It should be noted that 
some musical tones include formants. For example, for 
the vibration of a piano string, the fprmant frequency is 
shifted in consonance with the pitch. Since an input sig- 
nal is not limited to a voice, in the claims of the invention 



the term "timbre" is used as a concept that includes the 
above described sound quality. 
[0147] An appropriate machine to which the voice 
signal/tone signal processing apparatus of the invention 

5 can be applied is: an amusement apparatus such as an 
electronic musical Instrument, a game machine or a 
karaoke machine, that includes a function for entering a 
voice signal or a musical tone signal; various home 
appliances, such as a television; and a personal compu- 

10 ter. The processing apparatus of the Invention can be 
used as a voice signal/tone signal processor for these 
machines. 

[0148] As apparent from the above description, 
according to the present invention, the clear timbre 

15 change, various pitch conversions, and the application 
of sound effects to an input signal can be easily per- 
formed to generate a new voice signal based on the 
input voice. A variety of music performance effects can 
be provided, including a unique effect that can be added 

20 by making an adjustment that permits instant play, and 
a chorus having correct intervals can be provided by a 
single singer. In this manner, various music play effects 
can be obtained. 

25 Claims 

1. A music apparatus for receiving an input signal 
composed of either of a voice signal and a tone sig- 
nal and for processing said input signal based on a 

30 timbre change command signal to generate at least 
one channel of an output signal, the music appara- 
tus comprising: 

reference pitch designation means for desig- 

35 nating a reference pitch; and 

output signal generation means receptive of 
said input signal, said timbre change command 
signal and said reference pitch designated by 
said reference pitch designation means for 

40 Changing a timbre of said input signal in 

accordance with said timbre change command 
signal, and for changing a pitch of said input 
signal above or below said reference pitch in 
accordance with said timbre change command 

^5 signal, thereby generating the output signal 

having the changed timbre and the changed 
pitch. 

2. The music apparatus according to claim 1 . wherein 
so the output signal generation means changes the 

pitch of the input signal above the reference pitch 
when the timbre of the input signal is changed by 
converting an original formant of the input signal to 
a female formant, and the output signal generation 
55 means changes the pitch of the input signal below 
the reference pitch when the timbre of the input sig- 
nal is changed by converting an original fonnant of 
the input signal to a male formant. 
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3. A music apparatus for receiving an Input signal 
composed of either of a voice signal and a tone sig- 
nal and for processing said input signal in accord- 
ance with a timbre change command signal to 
generate at least one channel of an output signal, 5 
the music apparatus comprising: 

pitch detection means for detecting a pitch of 7. 
said input signal; and 

output signal generation means receptive of 10 
said input signal, said timbre change command 
signal and said pitch of said input signal that is 
detected by said pitch detection means for 
changing a timbre of said input signal based on 
said timbre change command signal and for is 
increasing or decreasing said pitch of said 
input signal based on said timbre change com- 
mand signal, thereby generating said output 
signal having the changed timbre and the 
changed pitch. 20 

4. The music apparatus according to claim 3, wherein 
the output signal generation means increases the 
pitch of the input signal when the timbre of the input 
signal is changed by converting an original fbnmant 2S 
of the input signal to a female formant, and the out- 
put signal generation means decreases the pitch of 

the input signal when the timbre of the input signal 
is changed by converting an original formant of the 
input signal to a male formant. 30 

5. A music apparatus for receiving an input signal 
composed of either of a voice signal and a tone sig- 
nal and for processing said input signal in accord- 8. 
ance with a chord designation signal to generate at 3S 
least one channel of an output signal, the music 
apparatus cpmprising: 

a pitch conversion table stored for use in con- 
version of a pitch according to a chord; 40 
pitch determination means receptive of at least 
the chord designation signal which designates 
a chord for referring to said pitch conversion 
table to determine a pitch of said output signal 
based on the designated chord; and 4S 
output signal generation means receptive of 
said input signal for changing a pitch of said 
input signal to the pitch determined by said 
pitch determination means thereby generating 
said output signal having the determined pitch, so 9. 

6. The music apparatus according to claim 5. compris- 
ing a plurality of pitch conversion tables corre- 
sponding to a plurality of hannony types which can 

be selected to determine a particular harmonic 55 
relation between said input signal and said output 
signal, wherein said pitch determination means 
refers to a pitch conversion table conresponding to 



the selected harmony type to determine a pitch of 
said output signal, and said output signal genera- 
tion means gen rates said output signal having the 
determined pitch in parallel to said input signal to 
establish the particular harmonic relation therebe- 
tween. , 

A music apparatus for receiving an input signal 
composed of either of a voice signal and a tone sig- 
nal and for processing said input signal in accord- 
ance with a kit designation signal to generate at 
least one channel of an output signal, the music 
apparatus comprising: 

memory means for storing a plurality of param- 
eter kits, each of which Is constituted by a plu- 
rality of parameters used for characterizing 
said output signal and each of which includes 
at least a parameter used for controlling a pitch 
of said output signal; 

parameter output means receptive of said kit 
designation signal that designates one of the 
parameter kits for referring to said designated 
parameter kit to output therefrom at least said 
parameter used for controlling the pitch of said 
output signal; and 

output signal generation means for receiving 
said input signal and for changing a pitch of 
said input signal based on at least said param- 
eter that is output by said parameter output 
means, thereby generating said output signal 
having the changed pitch. 

The music apparatus according to claim 7. wherein 
said memory means stores a plurality of parameter 
kits in correspondence to a plurality of harmony 
modes including a vocoder harmony mode, a 
chordal harmony mode, a detune harmony mode 
and a chromatic harmony mode, each of which is 
used for characterizing a harmonic relation of said 
output signal to said Input signal, said parameter 
output means refers to said designated parameter 
kit to output therefrom said parameters used for 
controlling said output signal, and said output signal 
generation means generates said output signal In 
parallel to said input signal to establish the har- 
monic relation therebetween according to the des- 
ignated parameter kit. 

A music apparatus for receiving an input signal 
composed of either of a voice signal and a tone sig- 
nal and for processing said input signal to generate 
at least one channel of an output signal, the music 
apparatus comprising: 

effect setting means for setting parameters that 
are related to one or more sound effects to be 
applied to said output signal; 
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effect instruction means for Instructing applica- 
tion of at least one of said sound effects; and 
effect applying means operative based on said 
paranneters that are set by said effect setting 
means and that are r lated to said sound effect s 
for processing said input signal to generate 
said output signal applied with said sound 
effect that is designated by said effect instruc- 
tion means. 

10 

10. The music apparatus according to claim 9, wherein 
said effect instruction means Is manually operable 
to instruct application of a sound effect to said out- 
put signal independently from said input signal, and 
said effect applying means generates said output is 
signal in parallel to said input signal while applying 
said sound effect designated by said effect Instruc- 
tion means to said output signal independently from 
said input signal. 

20 

11. A method of processing an input signal composed 
of either of a voice signal and a tone signal based 
on a timbre change command signal to generate an 
output signal, the method comprising the steps of: 

25 

designating a reference pitch for an output sig- 
nal; 

receiving said input signal and said tinnbre 
change command signal; 

changing a timbre of said input signal in 30 
accordance with said timbre change command 
signal; and 

changing a pitch of said input signal above or 
below said designated reference pitch in 
accordance with said timbre change command 35 
signal to thereby generate the output signal 
having the changed timbre and the changed 
pitch. 

12. A method of processing an input signal composed 40 
of either of a voice signal and a tone signal in 
accordance with a timbre change command signal 
to generate an output signal, the method compris- 
ing the steps of: 

45 

detecting a pitch of said input signal; 
receiving said Input signal and said timbre 
change command signal; 
changing a timbre of said input signal based on 
said timbre change command signal; and so 
increasing or decreasing said pitch of said 
input signal based on said timbre change com- 
mand signal to thereby generate said output 
signal having the changed timbre and the 
changed pitch. 55 

13, A method of proc ssing an input signal composed 
of either of a voice signal and a tone signal in 



accordance with a chord designation signal to gen- 
erate an output signal, the method comprising the 
steps of: 

providing a pitch conversion table for use in 
conversion of a pitch according to a chord; 
referring to said pitch conversion table based 
on a chorcl designated by said chord designa- 
tion signal to detenmine a pitch of said output 
signal; and 

changing a pitch of said Input signal to the pitch 
determined by use of said pitch conversion 
table to thereby generate said output signal 
having the detennlned pitch. 

14. A method of processing an Input signal composed 
of either of a voice signal and a tone signal in 
accordance with a kit designation signal to gener- 
ate an output signal, the method comprising the 
steps of: 

providing a plurality of parameter kits, each of 
- which is constituted by a plurality of parameters 
used for characterizing said output signal, and 
each of whfeh includes at least a parameter 
used for controlling a pitch of said output sig- 
nal; 

referring to one of said parameter kits desig- 
nated by said kit designation signal to rBtrieve 
said parameter from said designated parame- 
ter kit; and 

processing said input signal to change a pitch 
of said input signal based on said parameter 
retrieved from said designated parameter kit, 
thereby generating said output signal having 
the changed pitch. . 

15. A method of processing an input signal composed 
of either of a votee signal and a tone signal to gen- 
erate an output signal, the method comprising the 
steps of: 

setting parameters that are related to one or 
more sound effects to be applied to said output 
signal; 

instructing application of at least one of said 
sound effects; and 

processing said input signal based on said 
parameters that are set and that are related to 
said sound effect to generate said output signal 
which is applied with said sound effect upon 
instructing of the application of said sound 
effect 

16, A medium for use in a music apparatus having a 
CPU, said medium containing a computer program 
executable by said CPU for causing said musfe 
apparatus to perform a method of processing an 
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input signal composed of either of a voice signal 
and a tone signal based on a timbre change com- 
mand signal to generate an output signal, wherein 
the method comprises the steps of: 

5 

designating a reference pitch for an output sig- 
nal; 

receiving said input signal and said timbre 
change command signal; 

changing a timbre of said input signal in io 
accordance with said timbre change command 
signal; and 

changing a pitch of said input signal above or 
below said designated reference pitch in 
accordance with said timbre change command is 
signal to thereby generate the output signal 
having the changed timbre and the changed 
pitch. 

17. A medium for use in a music apparatus having a 20 
CPU, said medium containing a computer program 
executable by said CPU for causing said music 
apparatus to perform a method of processing an 
input signal composed of either of a voice signal 
and a tone signal in accordance with a timbre 25 
change command signal to generate an output sig- 
nal, wherein the method comprises the steps of: 

detecting a pitch of said input signal- 
receiving said input signal and said timbre 30 
change command signal; 
changing a timbre of said input signal based on 
said timbre change command signal; and 
increasing or decreasing said pitch of said 
input signal based on said timbre change com- 35 
mand signal to thereby generate said output 
signal having the changed timbre and the 
changed pitch. 

18. A medium for use in a music apparatus having a 40 
CPU, said medium containing a computer program 
executable by said CPU for causing said music 
apparatus to perform a method of processing an 
input signal composed of either of a voice signal 
and a tone signal in accordance with a chord desig- 45. 
nation signal to generate an output signal, wherein 

the method comprises the steps of: 

providing a pitch conversion table for use in 
conversion of a pitch according to a chord; so 
referring to said pitch conversion table based 
on a chord designated by said chord designa- 
tion signal to determine a pitch of said output 
signal: and 

changing a pitch of said input signal to the pitch ss 
determined by use of said pitch conversion 
table to thereby generate said output signal 
having the detemnined pitch. 



19. A medium for use in a music apparatus having a 
CPU, said medium containing a computer program 
executable by said CPU for causing said music 
apparatus to perform a method of processing an 
input signal composed of either of a voice signal 
and a tone signal in accordance with a kit designa- 
tion signal to generate an output signal, wherein the 
method comprises the steps of: 

providing a plurality of parameter kits, each of 
which is constituted by a plurality of parameters 
used for characterizing said output signal, and 
each of which includes at least a parameter 
used for controlling a pitch of said output sig- 
nal; 

referring to one of said parameter kits desig- 
nated by said kit designation signal to retrieve 
said parameter from said designated parame- 
ter kit; and 

processing said input signal to change a pitch 
of said input signal based on said parameter 
retrieved from said designated parameter kit, 
thereby generating said output signal having 
the changed pitch. 

20. A medium for use in a music apparatus having a 
CPU, said medium containing a computer program 
^ecutable by said CPU for causing said music 
apparatus to perform a method of processing an 
input signal composed of either of a voice signal 
and a tone signal to generate an output signal, 
wherein the method comprises the steps of: 

setting parameters that are related to one or 
more sound effects to be applied to said output 
signal; 

instructing application of at least one of said' 
sound effects; and 

processing said input signal based on said 
parameters that are set and that are related to 
said sound effect to generate said output signal 
which is applied with said sound effect upon 
instructing of the application of said sound 
effect. 
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